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Recent years have witnessed much progress in the incorporation of 
economic considerations into the evaluation of public health interven- 
tions. In England, the Centre for Public Health Excellence within the 
National Institute for Health and Care Excellence (NICE) works to 
develop guidance for preventing illness and assessing which public 
health interventions are most effective and provide best value for 
money} Chapter 6 of the recent publication Methods for the 
Development of NICE Public Health Guidance considers in detail the 
role that health economics can play.^ In the United States, the Centres 
for Disease Control and Prevention have produced a set of public 
health economics tools, including a highly readable summary of the 
relationship between economics and public health and a guide to 
assessing the cost-effectiveness of public health prevention initia- 
tives.^'"^ 

Value for money is a key tenet of the economic perspective: those 
charged with allocating spending to public health interventions should 
identify and fund those interventions which, evidence shows, offer the 
best value for money. The idea is that the benefits that are offered by 
an intervention should be worth the costs of providing it. The tradi- 
tional economic approaches to evaluating such programmes include 
cost-effectiveness and cost-utility analysis, which focus on the gain to 
health for the individuals directly affected by the intervention. 
However, as noted by NICE, public health interventions may require a 
wider set of outcomes to be measured. In England, from March 2013, 
local government has taken over responsibility for implementing pub- 
lic health programmes. As such, NICE notes, cost-consequence and 
cost-benefit analysis, which have a wider remit in terms of the out- 
comes that they measure,^ may sometimes be preferred. 

Given these encouraging developments, it is worth trying to identi- 
fy where future effort at the level of the design and implementation of 
economic evaluations of public health interventions could result in 
greater payoffs. Here we propose two areas: i) whether, in some situa- 
tions, it is better to stop an evaluation early, because the evidence that 
has been collected to date is deemed to be conclusive; ii) how those 
charged with evaluating public health interventions from the econom- 
ic perspective should deal with the problem of uncertainty surrounding 
estimates of cost-effectiveness. These two topics, though posed sepa- 
rately, are interlinked, since both concern questions of how to assess, 
and act upon, an evolving evidence-base as an evaluation progresses. 

Economic evaluations of public health interventions have their own 
benefits and costs. On the cost side, there are the costs of paying a 
research team to undertake research, write up the results for publica- 
tion, and so on. On the benefits side, the reduction in uncertainty sur- 
rounding the estimate of the cost-effectiveness of an intervention 
implies a lower probability of making the wrong decision and so allo- 
cating resources to interventions that do not offer good value for 
money. Other costs and benefits may be less apparent, however. For 
example, delaying to the wider public access to a new, cost-effective, 
approach to improving the public health while an evaluation progress- 
es, perhaps in order to achieve a pre-determined level of confidence in 
results, can impose a cost on society. 



Recent advances in the medical statistical literature permit the 
early termination of evaluations when evidence strongly suggests that 
a new intervention is more effective, or less effective, than its com- 
parator. In group sequential design, ^^'^ those assessing the intervention 
have the option to stop an evaluation early if the evidence that has 
accumulated is convincing enough to suggest that a new intervention 
should be adopted (or not, as the case may be). The savings in 
research costs can then be reallocated to other research projects, or to 
the interventions \h^m%^\M^%. Adaptive design permits flexibility in the 
allocation of participants in an evaluation as it progresses, with more 
participants being allocated to the preferred intervention based 
according to the evidence which has accumulated to date.^ Hendricks- 
Brown et al.^ discuss a series of adaptive approaches which could be 
used in the public health arena, and assess the opportunities and 
threats posed by them. 

The second area concerns the treatment of the uncertainty sur- 
rounding estimates of the cost-effectiveness of public health interven- 
tions. In its examples of evidence tables, NICE distinguishes between 
requirements for what it terms quantitative studies and economic eval- 
uations.^^ The evidence table for quantitative studies requires the 
reporting of all measures of statistical uncertainty, including confi- 
dence intervals, P-values, standard deviations and standard errors. The 
evidence table for economic evaluations classifies analysis of uncer- 
tainty surrounding estimates of cost-effectiveness as secondary; the 
primary outcomes to be reported concern the point estimates of 
expected cost-effectiveness. Further, NICE'S recommendations for 
whether or not a public health intervention is deemed cost-effective 
suggest that uncertainty only matters when the cost-per-QALY esti- 
mate exceeds £20,000: in general, interventions with an ICER of less 
than £20,000 per QALY gained are considered to be cost effective. For 
estimates that are greater than ^0,000 per QALY gained, other factors 
should be taken into account, such as the degree of uncertainty around 
the ICER} NICE therefore appears to treat the (quantitative) evidence 
arising from economic evaluations differently to quantitative evidence 
arising from other studies, such as clinical evaluations. 
Why is uncertainty surrounding a cost-effectiveness estimate, regard- 
less of any threshold, potentially important? First, estimates of cost- 
effectiveness can show large variability from one study to another and 
be sensitive to the methodology adopted. This is especially the case 
when decision modelling is used to obtain estimates of long-term 
health outcomes from a study with a short period of follow-up. 
Secondly, as noted by recent contributions to the literature,^^"^^ the 
study of uncertainty allows investigators to carry out so-called value 
of information (Vol) analyses in order to define research priorities. 
These analyses compare the expected benefit from narrowing the 
uncertainty in a study with the expected cost of obtaining it by 
increasing the sample size. Attention paid to Vol in NICE documenta- 
tion is relatively limited, but is likely to grow. Finally, uncertainty mat- 
ters because of the existence of potentially irreversible consequences 
associated with making public health decisions. For example, there 
could exist costs of adopting a new intervention which, once spent. 
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cannot be recovered (examples include training staff and building new 
premises). In such cases, when an evidence base evolves over time, a 
decision concerning whether to carry out a new intervention to 
improve the public health cannot be separated from that about when it 
is best to do so. 
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